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The differences between open-book, cheat sheet, and closed-book exams were examined in two different types of 
psychology courses. A total of 297 students enrolled in eight sections of Introductory Psychology and 99 students 
enrolled in four sections of Statistics participated in this study. Exam types were counterbalanced across sections of 
the same course. Students were given either open-book, cheat sheet, or closed-book exams, took a surprise quiz two 
weeks after the exams to measure retention of course material, completed a preference questionnaire, and took a 
pre-test measure of test anxiety on open-book and cheat sheet tests. While students did slightly better on open-book 
exam than on closed-book exams, they also much preferred open-book and cheat sheet exams over closed-book 
exams, and had lower levels of anxiety when taking open-book exams compared to cheat sheet exams. Based on 
these results, open-book exams may be a superior style of examination for a variety of psychology courses. 

Keywords: test anxiety, test style, student assessment, academic performance, open-book tests, closed-book tests 

This study developed from an ongoing debate in our department, one which is probably happening in 
many academic departments. The question is which type of exam is best. All teachers want their students to 
learn and retain the material they cover and at the same time enjoy the process of learning. One way to check 
how well students have learned is by giving exams. There are many teachers who consider the traditional 
closed-book and notes time-limited exam the best measure of student learning, while others argue for 
alternatives, such as cheat sheet or open-book tests. While a closed-book exam is probably the most common 
type of exam in psychology courses, alternative exam types, such as cheat sheets (or crib notes) where the 
students can prepare notes to use on the exam ahead of time, and open-book exams where the student can use 
all their material during the test, are increasingly accepted. Students themselves prefer alternative forms of 
testing, for example, Williams and Wong (2009) found that students prefer open-book exams compared to 
closed-book tests. Students believed that being able to use notes, cheat sheets and texts during an exam would 
improve grades and decrease their anxiety. Are these expectations justified? The purpose of this project was to 
examine the effectiveness of various exam types: open-book, closed-book, and cheat sheet. There is a relatively 
long history of trial-and-error efforts to improve testing procedures (Kalish, 1958; J. M. Stalnaker & R. C. 
Stalnaker, 1934). Surprisingly, there is little consensus among investigators as to which testing style may be the 
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most effective. 

J. M. Stalnaker and R. C. Stalnaker (1934) suggested that while the closed-book exam is the norm in 
higher education, in fact, an open-book test may be preferable since it puts less emphasis on memorization of 
facts and encourages a deeper engagement with the course material on the part of the students. Similarly, 
Eilertsen and Valdermo (2000) argued that an open-book exam encourages greater engagement and improves 
understanding of course material. Feller (1994) further suggested that the open-book exam is superior to 
closed-book exams as it is more realistic — similar to problem-solving situations students are likely to face 
outside of academia. And of course, students prefer open-book to closed-book exams and find them less 
stressful (Philips, 2006). As Kalish (1958) pointed out, however, open-book test may encourage less studying. 

The evidence for the benefits of open-book test is mixed. Kalish (1958) found that grades do not improve 
on open-book tests compared to closed-book exams, but other researchers have found higher grades on 
open-book tests, although the improvement is more modest than what might be expected (Krarup, Naeraa, & 
Olsen, 1974). Students who are struggling with the course material are particularly likely to see an 
improvement in their grades on an open-book exam (Philips, 2006). Those higher grades do not necessarily 
mean better learning of the material. For example, Agarwal, Karpicke, Kang, Roediger, and McDermott (2008) 
found that while students performed better on an open-book test, there was no difference on a later retention 
test between open- and closed- book tests. In observing students taking open-book examinations, Boniface 
(1985) found that weaker students relied more on their notes and texts than academically stronger students, and 
students that used their notes most ended up doing more poorly on the exam, suggesting that open-book exams 
may actually lead to lower scores. Moore and Jensen (2007) found that while exam scores in an Introductory 
Biology class were higher on open-book tests, the open-book format also encouraged lower attendance and less 
effort (fewer extra credit assignments and lower attendance at review sessions), and lower retention of 
information over the long tern. Heijne-Penninga, Kuks, Hofman, and Cohen-Schotanus (2008) found that in 
medical students, a closed-book exam actually encourages more in-depth engagement with the course material. 

An alternative to either closed-book or open-book exams is a cheat sheet exam, where the student is given 
the opportunity to prepare a sheet of notes ahead of time to use on the exam. Erbe (2007) had argued that the 
preparation of cheat sheets improve and deepen learning by helping students organize their study time. Some 
authors found an improvement in performance when students use cheat sheets (Skidmore & Aagaard, 2004), 
while others found no effect (Dickson & Miller, 2005; Hindman, 1980; Whitley, 1996). Funk and Dickson 
(2011), and Dickson and Bauer (2008) argued that cheat sheets are more of a crutch than an aid in preparing for 
an exam and found that students who prepared cheat sheets for an exam showed no improvement in learning on 
a pre-test without their cheat sheets. 

One important benefit of alternatives to closed-book exams is that they may lower test anxiety. Test 
anxiety — worrying about the outcome of the test and experiencing negative emotions during the test — is 
associated with poor performance on exams (Morris, Davis, & Hutchings, 1981; Tryon, 1980). Students prefer 
open-book and cheat sheet exams because they feel they would be less anxious during those types of exams and 
therefore perform better (Zoller & Ben Chaim, 1988). Theophilides and his colleagues (Theophilides & 
Koutselini, 2000; Theophilides & Dionysiou, 1996) found that students reported that they were more optimistic 
and less anxious when taking open-book exams. On the other hand, Dickson and Miller (2005) found that 
students did not feel that being able to use cheat sheets decreased their test anxiety, although in their study, 
students reported their impressions at the end of the term, not during the exam itself. 
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Some of the discrepancy in findings about the effectiveness of various exam types may be due to 
differences between classes and disciplines. While some of the research reviewed above focused on psychology 
courses (e.g., Kalish, 1958; Dickson & Bauer, 2008; Whitely, 1996), others used students in science and 
mathematics classes (e.g., Zoller & Ben Chaim, 1988; Phillips, 2006; Moore & Jensen, 2007), and teacher 
education classes (e.g., Skidmore & Aagaard, 2004). Some of the classes were lower division, others upper 
division. It is likely that different types of courses demand different styles of studying and test for different 
types of material. In some courses, taking an Introductory Psychology course for example, studying for an 
exam involves reviewing facts. For other classes, such as Mathematics or Statistics, the student may need to 
practice using formulas and calculations. The most effective type of test — in terms of both performance and 
retention — is likely to be different in the two types of courses. In addition, one type of test may lead to higher 
test scores, but less long-term retention of material, another may lead to a less stressful testing environment but 
lower test scores. To our knowledge, there have been no attempts to directly compare open-book, cheat sheet, 
and closed-book exams in lower and upper division psychology courses. 

In the present study, the effectiveness of different exam types (open-book, closed-book, and cheat sheet) 
were compared in two different types of classes, an Introductory Psychology class typically taken by freshmen, 
and a Statistics course typically taken by juniors. Students in the Introductory class were given an open-book 
exam, cheat sheet exam, and closed-book exam. As students were not required to memorize statistical fonnulas 
in their Statistics class, closed-book exams were not practical in that course, instead only open-book and cheat 
sheet exams were compared. The effectiveness of exams was measured by comparing exam scores, asking 
students about their preferences, measuring long-term retention of material on quizzes several weeks after the 
exam, and measuring anxiety levels immediately before the exam. It was hypothesized that exam scores would 
be the highest when taking open-book and cheat sheet exams, retention of material would be the highest after 
closed-book and cheat sheet exams, and test anxiety would be the lowest during open-book exams. 

Method 

Participants 

A total of 396 undergraduate students enrolled at a small liberal arts university participated in this study. 
There were 297 (64 males and 233 females) students enrolled in eight different sections of Introductory 
Psychology, and 99 (12 males and 87 females) participants enrolled in four sections of Statistics. This project 
was approved by the university’s Institutional Review Board for the Protection of Human Subjects. 

Materials 

In all sections of both Introductory Psychology and Statistics, students completed a 3 -item Exam 
Preference Questionnaire, which asked participants which type of test (open-book, cheat sheet, or closed-book) 
they predicted that they would do their best on, which type of test they would spend the most time studying for 
and which type of test they preferred to take. 

On cheat sheet exams, both Introductory Psychology and Statistics participants were instructed to prepare 
a double-sided 8!4 x 11 sheet of paper on which they could write as much information as they wanted to use on 
the exam. These cheat sheets were collected and scored by an independent scorer on a 10-point scale for 
organization and richness of detail. 

In five of the Introductory Psychology sections and two of the Statistics sections, the students completed 
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the Pre-examination Worry-Emotionality Scale (Morris, Davis, & Hutchings, 1981), a 10-item measure of test 
anxiety that asks (on a 5-point Likert scale) questions about current levels of negative emotionality (“I feel my 
heart beating fast”) and worry (“I am afraid that I should have studied more for this test”). The overall scores 
were used as a measure of test anxiety in this study. 

A 10-item quiz (all multiple choice) was used to measure how much students remembered from the exams 
in both courses. The items on the quiz covered the same material as the previous exam, but were not the 
identical to exam questions. 

Procedure 

In Introductory Psychology, all sections were taught by the same instructor, had the same texts and 
assignments, and during a given term, identical exams (non-cumulative, 50 multiple-choice questions on each 
exam) were given to all sections. The sections of Introductory Psychology were taught over four terms (three 
sections the first term, two sections the next two terms, and one section the fourth term). 

In Statistics, all sections were taught by the same instructor, and had the same textbook, homework 
assignments, and exams. There were two sections taught each term for two terms. There were two exams, and 
the exams were a combination of short answer and story problems. 

Exam performance. Student performance on closed-book exams was compared to perfonnance on cheat 
sheet and open-book tests in three sections of Introductory Psychology, all taught the same tenn. The three 
exams were counterbalanced among the three sections and the three exam types (open-book, cheat sheet, and 
closed-book), so that each exam was given in all three formats (to different sections). 

In the remaining five sections of Introductory Psychology, the first two exams were either open-book or 
cheat sheet (counterbalanced across sections in four sections, and open-book as the first and cheat sheet as the 
second in the final section). The cheat sheets were collected in these sections and scored for organization and 
detail on a 10-point scale. 

In Statistics, the two exams were counterbalanced among the four sections and two exam types 
(open-book and cheat sheet). Cheat sheets were collected and scored for organization and detail one term (two 
sections). 

Student preferences. After the first exam in each section of class (both in Introductory Psychology and 
Statistics), the students completed the Exam Preference Questionnaire, and also reported the number of hours 
that they had spent on studying for that exam. 

Retention of material. Two weeks after the first two exams in five sections of Introductory Psychology 
and in all four sections of Statistics, an unannounced closed-book retention quiz was given. This quiz was 
anonymous (The students did not put their names in the quiz). 

Test anxiety. In five sections of Introductory Psychology (none of which included a closed-book exam 
condition), and in two sections of Statistics students completed the Morris et al.’s (1981) Pre-examination 
Worry-Emotionality Scale immediately before taking their first two exams (either open-book or cheat sheet). 

Results 

Exam Scores 

Table 1 shows exam scores and retention quiz scores for both the classes. In Introductory Psychology, an 
ANOVA (analysis of variance) found differences between scores on the three different types of exams (Fp, do) 



EFFECTS OF EXAM TYPES ON PERFORMANCE, RETENTION, AND ANXIETY 


473 


= 7.74, p < 0.05). Paired-samples I-tests revealed that closed-book exams resulted in lower scores than either 
open-book exams (7(68) = 4.00, p < 0.05) or cheat sheet exams (1(67) = 2.43, p < 0.05), and cheat sheet exams 
resulted in lower scores than open-book exams (1(286) = 3.45, p < 0.05). In Statistics, the difference in exam 
scores between open-book and cheat sheet exams was not significant (1( 98 ) = 1.98, p > 0.05) . 

Table 1 


Exam and Quiz Scores 


Class 

Open-book 

Cheat sheet 

Closed-book 

Exam scores 

80.11 (10.92) 
N= 287 

77.85 (12.14) 
N= 287 

72.52(11.81) 
N =69 

Introductory Psychology 

Quiz scores 

6.41 (1.94) 
N= 78 

6.44(1.88) 
N= 79 

6.38(1.94) 

1V=34 

Exam scores 

Statistics 

80.73 (13.05) 
N= 99 

77.48 (14.89) 
N= 99 


Quiz scores 

6.18(1.75) 
N= 169 

6.31 (1.83) 
N= 174 



Notes. Values are means (and standard deviations in parentheses) for the different types of exams calculated across all sections. 
For the exam scores, N reflects the number of students in all the sections of the two classes that took the exams; For the quiz 
scores, N reflects the total number of quizzes returned. 
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Figure 1. Correlations between different exam types across all sections and both courses. (A) Scores on open-book 
and cheat sheet exams ( N = 396); (B) Scores on closed-book and cheat sheet exams ( N = 67); (C) Scores on 
closed-book and open-book exams (N= 69). 
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Figure 1 shows the overall correlations between the three exam types. Pearson correlations reveal that 
scores on different types of exams are positively correlated in both classes. In Introductory Psychology, there is 
a positive correlation between scores on open-book and cheat sheet exams (r ( 2 85 ) = 0.54, p < 0 .05), open-book 
and closed-book (r (6 p = 0.66, p < 0.05), and cheat sheet and closed-book scores (r (6 6> = 0.67, p < 0.05). In 
Statistics, there is a positive correlation scores on open-book and cheat sheet exams (r ( 97) = 0.32, p < 0.05). 
Students who do well on one type of exam do well on the other two types. 

Cheat Sheets 

The quality of the cheat sheets prepared by students was rated on a 1 0-point scale on the organization of 
the cheat sheet (whether there are headings and subheadings, highlights, etc.) and amount of detail (number of 
words) by a rater blind to the students’ exam scores. A Pearson correlation found that the quality of the cheat 
sheet was not related to scores on cheat sheet exams in Introductory Psychology, but there was a positive 
correlation between cheat sheet quality and cheat sheet exam score in Statistics (ty 43 ) = 0.34, p < 0.05). 

Retention Quizzes 

As it can be seen in Table 1, there were no differences in retention quiz perfonnance across exam types in 
either course: In Introductory Psychology, an ANOVA comparing quizzes after open-book, cheat sheet, and 
closed-book exams finds F (2 , 105 ) = 0.03, p > 0.05; In Statistics, an independent samples t-test comparing 
quizzes given after open-book and cheat sheet exams finds hi 84 ) = 0.86, p > 0.05. 

Test Anxiety 

A Pearson correlation shows that the test anxiety measured right before the exam was negatively 
correlated with scores on the exam (r ( 5 2 g) = -0.15, p < 0.05). Figure 2 shows the differences in anxiety level 
between open-book and cheat sheet for the two classes. A paired-samples f-test finds higher anxiety scores 
during cheat sheet exam compared to open-book exam in both Introductory Psychology (t( 220 ) = 1 .98, p < 0.05) 
and Statistics (f ( 46 ) = 4.87, p < 0.05). 


t( 46)=4.87 



Class 


1 Open-Book 
I Cheat Sheet 


Figure 2. Mean and standard error of anxiety scores in Introductory and Statistics courses. Values are the total scores 
for the Worry-Emotionality scale. Paired-samples t-tests compared the two sets of scores. Differences are significant at 
p < 0.05. 


Student Preferences 


Table 2 summarizes the results of the student preferences questionnaire. As it can be seen, the students in 
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both classes predicted that they would do better on open-book or cheat sheet exams compared to closed-book 
exams (% ( 3 ) = 238.67, p < 0.05). Comparisons of the actual exam scores of those who predicted they would do 
best on open-book exams ( N = 1 92) and those who predicted they would perform best on cheat sheet exam ( N = 
136) were made. Those who predicted that they would do best on open-book exam scored an average of 79.3% 
(SD = 15.0) on open-book exam and 77.0% on cheat sheet exam ( SD = 16.3), those who predicted that they 
would do best on cheat sheet exam scored an average of 80.2% (SD = 12.0) on open-book exam and 76.6% (SD 
= 13.3) on cheat sheet exam. An independent samples f-test finds no difference between the two groups of 
students with scores of the two types of exams. 

Table 2 


Student Preference Questionnaire 




Introductory Psychology (%) 

Statistics (%) 

Overall (%) 


Closed-book 

6.3 

2.4 

5.4 


Open-book 

53.0 

47.1 

51.6 

Predicted best 

33.8 

48.2 

37.1 

Cheat sheet 


No preference 

7.0 

2.4 

5.9 


Closed-book 

76.0 

65.9 

73.7 

Study time 

Open-book 

5.6 

4.7 

5.4 

Cheat sheet 

6.6 

9.4 

7.3 


No preference 

11.8 

20.0 

13.7 


Closed-book 

7.3 

2.4 

6.2 


Open-book 

51.7 

41.7 

49.5 

Preferred exam 

35.0 

54.8 

39.5 

Cheat sheet 


No preference 

5.9 

1.2 

4.9 


Notes. Values are percent of participants reporting a preference for that exam type. Students were asked before the first exam to: 
(1) predict which exam type they would do best on; (2) which exam type they think they would study most for; and (3) which 
exam type they preferred. N= 372 total (287 participants in Introductory Psychology and 85 in Statistics). 


Students believed that they would study most for closed-book exams (/ 2 q) = 475.38, p < 0.05). Students 
were asked on the first exam to report how much time they had actually spent on studying for that test. For 
Introductory Psychology, the average study time reported for open-book exams is 3.97 hours (SD = 4.42, N = 
160), for cheat sheet exams study time is 4.04 hours (SD = 3.65, N = 103), and for closed-book exams reported 
study time is 3.32 hours (SD = 3.32, N = 23). Independent-samples t-tests find no differences between exam 
types in reported study time. In Statistics, students reported spending an average of 2.60 hours (SD = 1.66, N = 
36) on the open-book exam and 3.77 hours (SD = 6.86, N = 49) on the cheat sheet test. This difference is also 
not significant. 

Finally, students were asked which exam of the three exam types they would prefer to take. As it can be 
seen in Table 2, the students in both classes preferred open-book and cheat sheet exams over closed-book test 
(X 2 o') = 231.71, p < 0.05). To see if preferences are related to exam performance, the exam scores of those who 
preferred open-book exam (N = 183) were compared to those who preferred cheat sheets (N= 144). Those who 
stated a preference for open-book exam scored an average of 78.6% (SD = 14.4) on open-book test, and 76.0% 
(SD = 16.3) on cheat sheet tests. Those who preferred cheat sheet exams scored an average of 80.7% (SD = 
13.4) on open-book tests and 77.5% (SD = 13.2) on cheat sheet exams. An independent samples /-test reveals 
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no difference between the two groups of students on exam scores. 

Discussion 

The hypotheses were only partially supported. While exam scores were higher on open-book and cheat 
sheet exams compared to closed-book exam, and anxiety is the lowest during open-book exam, there were no 
differences in scores on retention quizzes. 

Scores among exam types are positively correlated — students who do well on one exam type tend to do 
well on the others. While open-book exams result in the highest scores and both open-book and cheat sheet 
exams result in a small improvement in exam scores compared to closed-book exams in Introductory 
Psychology classes, the difference is quite modest given students’ perceptions of which exams they would 
prefer and which exams they would do best on. In fact, students overestimated the advantage they would have 
on open-book and cheat sheet exams — those who predicted they would do better on one type of exam or the 
others or who had a preference for one type of exam over others actually did the same on both types of tests. To 
our surprise, cheat sheet exam scores are not significantly higher than open-book exam scores in Statistics 
courses, although the difference approaches significance and is in the same direction as in Introductory 
Psychology. Scores on the retention quizzes are not different between exam types, but anxiety levels are lower 
during open-book exam compared to cheat sheet exam. These results are in agreement with those researchers 
who have failed to find improved learning with alternative exam types (Kalish, 1958; Dickson & Miller, 2005; 
Flindman, 1980; Whitley, 1996). It would be interesting, in future studies, to include other alternative exam 
types, such as take-home exam and cooperative exam. It may be that there are some types of exams, not 
examined here, which do result in superior learning and retention. In addition, future research could look at the 
effects of various test preparation methods, such as study guides and review sessions, on exam performance and 
anxiety. 

The relative lack of effectiveness of cheat sheets compared to open-book exam in Statistics compared to 
Introductory Psychology may be explained in terms of the differences in course content and the material 
covered in exams. In Introductory level classes, students are tested on facts which the students may not have 
included on their cheat sheet, and their performance may therefore be improved on open-book exam. In 
Statistics, it is likely that the material included on a typical and well-constructed cheat sheet would be the same 
material that the students would look up in the text book (e.g., formulas), and therefore, there is little 
improvement when students are allowed their texts. This may also explain why the quality of the cheat sheet is 
positively correlated with grades in Statistics but not in Introductory classes. 

In tenns of test anxiety, higher levels of anxiety were associated with poorer performance on tests, and 
students were more anxious when taking cheat sheet tests. This may seem at first to be contrary to the findings 
of Dickson and Miller (2005), who found that cheat sheets did not decrease anxiety levels. In the case of 
Dickson and Miller (2005), however, the comparison was between cheat sheet exams and closed-book tests. 
Others have found that open-book exams decrease anxiety compared to closed-book tests (Theophilides & 
Dionysiou, 1996; Theophilides & Koutselini, 2000; Zoller & Ben-Chaim, 1988). The current results suggest 
that an open-book exam is also less anxiety provoking than a cheat sheet exam. Another difference between the 
present results and those of Dickson and Miller (2005) is that in the Dickson and Miller’s study, participants 
rated their anxiety at the end of the tenn (presumably after they received their grades on the exams) rather than 
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having their anxiety measured at the time of the exams. It could be that in retrospect and in light of the modest 
positive effects of cheat sheet exam on grades, the students may not remember cheat sheet exam as being 
significantly less anxiety provoking than open-book tests, even though they were at the time. This difference 
between anxiety levels during the exam and ones retrospective evaluation of anxiety would be an interesting 
question to further investigate. 

Agarwal, Karpicke, Kang, Roediger, and McDermott (2008) similarly found that open-book tests 
resulted in increases in performance on an initial test compared to a closed-book test, but that the two types 
of tests resulted in similar long-term retention, and both types of exams resulted in better long-term retention 
compared to simply studying material and not being tested on it. This suggests that if the main interest of 
instructors is that students retain course material, open-book, cheat sheet, and closed-book tests are equally 
effective. 

However, given the detrimental effect that test anxiety has on performance, and the preference students 
have for alternatives to closed-book exams, and given the strong correlation between scores on different 
exam types, it can be argued that open-book and cheat sheet exams are superior to the traditional closed-book 
test in a variety of psychology courses — All three types of exams are equally effective as teaching tools, are 
equally able to distinguish stronger from weaker students, and result in equal levels of studying and retention, 
but alternatives to closed-book exams decrease anxiety and give students the illusory belief that they will 
perform better. In addition, the finding that open-book exams cause less test anxiety than cheat sheet exams 
should encourage psychology teachers to consider open-book exams as an alternative to closed-book 
tests — An open-book exam still distinguishes good students from poor ones, is preferred by the students, 
does not seem to decrease learning and retention, and decreases anxiety levels. 
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